Some of us can't even get married ONCE!

Met a guy online again. He has a career, a house, a dog, and green eyes.
We spent the last two weekends together. I took him to work to meet animals, and to my house to meet my roommates. Everyone thinks he's great. I kept thinking, "why does he have to touch me on the small of my back everytime I walk through a door?" and "hey! why didn't he do it that time?!" I don't know if life is actually confusing or if it's just my inner dialog complicating things.
We told eachother our life stories. I told him things I haven't talked about in a long time because I like to pretend they don't exist. He told me things we would both like to pretend didn't exist, but for different reasons.

Here's our conversation:
Me: That's my life story, your turn.
Guy: You aren't judgemental, are you?
My Inner Dialog: HA! I bet he's been married.
Me: No. (I can only use small words when my inner dialog is talking, or I get confused and can't form complete sentences.)
Guy: When I was 21 I met a girl..blah blah blah...we got married.
M.I.D.: SNAP! I was right! Don't make a face... Maybe there was a good reason they split up.
Me: What happened?
Guy: Blah blah, *good reason*
Me: So then what?
Guy: Well, two years later, I met a girl...
M.I.D.: Holy Schnikies. He's gonna say it.
Guy: ...we got married.
Me: *LAUGHTER*
M.I.D.: *MANIACAL LAUGHTER*

I don't even remember much more of the conversaion. I apologized for laughing. He explained the reasons for his second divorce (which didn't sound like reasons at all), and I tried not to look like I wished he would get tired and go home. He told me to take all the time I needed to think about it, but he really wants to go out again. I was pretty much done thinking, but to be fair, I put in an extra day of thinking. Now I am really sure.

How do you call a guy and say "sorry, you seem really great, but I don't want to be anyone's 3rd wife." I've never really been in this position before.

Stupid internet. When, oh when will I learn?